69 research outputs found
Hierarchical Relational Learning for Few-Shot Knowledge Graph Completion
Knowledge graphs (KGs) are known for their large scale and knowledge
inference ability, but are also notorious for the incompleteness associated
with them. Due to the long-tail distribution of the relations in KGs, few-shot
KG completion has been proposed as a solution to alleviate incompleteness and
expand the coverage of KGs. It aims to make predictions for triplets involving
novel relations when only a few training triplets are provided as reference.
Previous methods have mostly focused on designing local neighbor aggregators to
learn entity-level information and/or imposing sequential dependency assumption
at the triplet level to learn meta relation information. However, valuable
pairwise triplet-level interactions and context-level relational information
have been largely overlooked for learning meta representations of few-shot
relations. In this paper, we propose a hierarchical relational learning method
(HiRe) for few-shot KG completion. By jointly capturing three levels of
relational information (entity-level, triplet-level and context-level), HiRe
can effectively learn and refine the meta representation of few-shot relations,
and consequently generalize very well to new unseen relations. Extensive
experiments on two benchmark datasets validate the superiority of HiRe against
other state-of-the-art methods.Comment: 10 pages, 5 figure
An Intelligent Trade Matching System for B2B Marketplace
With the fast growth of B2B sales, an intelligent system is greatly useful for decreasing transaction cost and increasing market efficiency on electronic platforms. In order to improve the quality of transaction processing and customer experience, this paper proposes a knowledge-based system, which employs a Case-Based Reasoning (CBR) technique for trade matching in B2B marketplace as a substitute for the manual matching process. The system function and logical architecture are discussed. And the case repository is proposed to support this CBR approach where the case representation, case base indexing, case base decomposition and the dictionary are argued in details
Efficient Vision Transformers via Fine-Grained Manifold Distillation
This paper studies the model compression problem of vision transformers.
Benefit from the self-attention module, transformer architectures have shown
extraordinary performance on many computer vision tasks. Although the network
performance is boosted, transformers are often required more computational
resources including memory usage and the inference complexity. Compared with
the existing knowledge distillation approaches, we propose to excavate useful
information from the teacher transformer through the relationship between
images and the divided patches. We then explore an efficient fine-grained
manifold distillation approach that simultaneously calculates cross-images,
cross-patch, and random-selected manifolds in teacher and student models.
Experimental results conducted on several benchmarks demonstrate the
superiority of the proposed algorithm for distilling portable transformer
models with higher performance. For example, our approach achieves 75.06% Top-1
accuracy on the ImageNet-1k dataset for training a DeiT-Tiny model, which
outperforms other ViT distillation methods
One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation
Knowledge distillation~(KD) has proven to be a highly effective approach for
enhancing model performance through a teacher-student training scheme. However,
most existing distillation methods are designed under the assumption that the
teacher and student models belong to the same model family, particularly the
hint-based approaches. By using centered kernel alignment (CKA) to compare the
learned features between heterogeneous teacher and student models, we observe
significant feature divergence. This divergence illustrates the ineffectiveness
of previous hint-based methods in cross-architecture distillation. To tackle
the challenge in distilling heterogeneous models, we propose a simple yet
effective one-for-all KD framework called OFA-KD, which significantly improves
the distillation performance between heterogeneous architectures. Specifically,
we project intermediate features into an aligned latent space such as the
logits space, where architecture-specific information is discarded.
Additionally, we introduce an adaptive target enhancement scheme to prevent the
student from being disturbed by irrelevant information. Extensive experiments
with various architectures, including CNN, Transformer, and MLP, demonstrate
the superiority of our OFA-KD framework in enabling distillation between
heterogeneous architectures. Specifically, when equipped with our OFA-KD, the
student models achieve notable performance improvements, with a maximum gain of
8.0% on the CIFAR-100 dataset and 0.7% on the ImageNet-1K dataset. PyTorch code
and checkpoints can be found at https://github.com/Hao840/OFAKD
Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism
In the past years, YOLO-series models have emerged as the leading approaches
in the area of real-time object detection. Many studies pushed up the baseline
to a higher level by modifying the architecture, augmenting data and designing
new losses. However, we find previous models still suffer from information
fusion problem, although Feature Pyramid Network (FPN) and Path Aggregation
Network (PANet) have alleviated this. Therefore, this study provides an
advanced Gatherand-Distribute mechanism (GD) mechanism, which is realized with
convolution and self-attention operations. This new designed model named as
Gold-YOLO, which boosts the multi-scale feature fusion capabilities and
achieves an ideal balance between latency and accuracy across all model scales.
Additionally, we implement MAE-style pretraining in the YOLO-series for the
first time, allowing YOLOseries models could be to benefit from unsupervised
pretraining. Gold-YOLO-N attains an outstanding 39.9% AP on the COCO val2017
datasets and 1030 FPS on a T4 GPU, which outperforms the previous SOTA model
YOLOv6-3.0-N with similar FPS by +2.4%. The PyTorch code is available at
https://github.com/huawei-noah/Efficient-Computing/tree/master/Detection/Gold-YOLO,
and the MindSpore code is available at
https://gitee.com/mindspore/models/tree/master/research/cv/Gold_YOLO.Comment: Accepted by NeurIPS 202
- …